fix(elastic+pipeline): keystore-backed cluster credentials + ProcessorsEquals self-compare - #407
Merged
Conversation
medcl
force-pushed
the
fixes/keystore-and-reload
branch
from
August 25, 2026 02:29
dfb4a45 to
37d2be9
Compare
SecretString's MarshalJSON emits the shadow mask for plain-text values, so a cluster saved through the ORM (the /easysearch/ CRUD) never persisted its real basic_auth password or token — every consumer that loaded the record back (boot-time live registration, the cluster-change hook, app-side resolvers) authenticated with the mask and got 401s from secured clusters. - StashClusterSecrets/FromDelta store plain credentials under cluster-scoped keystore keys before the ORM write; switching auth mode drops the other key so hydration cannot resurrect a stale secret - HydrateClusterSecrets fills masked/missing credentials wherever an ORM-loaded record is used (CRUD post hooks, cluster-change hook, LoadClustersFromORM); in-memory real values always win - RemoveClusterSecrets cleans up on delete; keystore.DeleteValue added - ORM records keep the mask, so API responses never leak the secret
targetCfg was built from this instead of target, so Equals always saw identical processor configs and config-file updates never reloaded pipeline tasks — every change required a process restart to take effect (stale ship endpoints, stale harvest patterns). Regression test covers a nested ship_config.endpoints change.
medcl
force-pushed
the
fixes/keystore-and-reload
branch
from
August 25, 2026 03:04
37d2be9 to
2b9d549
Compare
SteveLauC
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two framework fixes found and verified in live LogPilot/Gateway use (full multi-instance isolation testing).
1. Cluster credentials never survived the ORM (
fix(elastic))SecretString'sMarshalJSONemits the shadow mask for plain-text values, so a cluster saved through the ORM (the/easysearch/CRUD) never persisted its real basic_auth password or token — every consumer that loaded the record back (boot-time live registration, the cluster-change hook, app-side resolvers like LogPilot's stream search) authenticated with the mask and got 401s from secured clusters. The health-status persist loop made it worse: identity comparison (real in-memory vs masked ORM record) re-registered the live client with the mask.StashClusterSecrets/FromDeltastore plain credentials under cluster-scoped keystore keys before the ORM write; switching auth mode drops the other key so hydration cannot resurrect a stale secretHydrateClusterSecretsfills masked/missing credentials wherever an ORM-loaded record is used (CRUD post hooks, cluster-change hook,LoadClustersFromORM); in-memory real values always winRemoveClusterSecretscleans up on delete;keystore.DeleteValueadded2. Pipeline config reload never fired (
fix(pipeline))ProcessorsEqualsbuilttargetCfgfromthisinstead oftarget— it compared the source config with itself, soEqualsalways saw identical processors and config-file updates never reloaded pipeline tasks. Every change (ship endpoints, harvest patterns) required a process restart. Regression test covers a nestedship_config.endpointschange.Verification
core/elastic(cluster-secrets ORM round-trip: stash → masked marshal → hydrate → retyped password → update → delete),core/pipeline(nested-change detection),modules/elastic,modules/easysearchNote: two sibling fixes that touch
modules/configs/{server,reverseclient}(pending-instance sync wiping local configs; loopback rewrite breaking specific-IP bindings) are added to #405 since that package only exists there.